This document provides a summary of the change-point analysis for Blastomycosis (blasto).

Piecewise CP Approach

The following section summarizes change-point analysis for trends in the number of visits before diagnosis using the standard piecewise-modeling approach to find the change-point. Specifically, we evaluate 4 peicewise models with linear, quadratic, cubic and exponential trends. The change-point is found by iterating over different change-points and selecting the best fitting model based on AIC.

Summary of Optimal Models based on RMSE

The following table depicts the optimal change-point selected for each approach based on RMSE.
label Change Point Pred. Bound CP CP # Miss PB CP # Miss
Piecewise lm w/ periodicity 63 71 6350.29 6477.83
Piecewise quad w/ periodicity 77 51 4986.88 4786.87
Piecewise cubic w/ periodicity 84 51 3880.73 3767.81
Piecewise exp w/ periodicity 49 51 3675.31 3708.80
Piecewise lm 63 67 6350.52 6399.20
Piecewise quad 77 51 4988.12 4737.32
Piecewise cubic 84 50 3939.88 3757.99
Piecewise exp 49 51 3670.72 3696.93

The following figure depicts the optimal change-point for each method with periodicity.

The following figure depicts the optimal change-point for each method without periodicity.

Summary of Model Performance Across Change-points

The following table summarizes RMSE and the implied number of missed opportunities across each method for the different change-points evaluated
Change Point
Linear
Quadratic
Cubic
Exponential
weeks days RMSE N Miss Visits RMSE N Miss Visits RMSE N Miss Visits RMSE N Miss Visits
1 7 33.388 1082 21.005 655 14.327 344 18.069 160
2 14 27.829 2068 17.881 1317 13.075 757 17.004 570
3 21 23.823 2925 15.907 1921 12.377 1152 16.048 1064
4 28 20.155 3751 13.993 2578 11.699 1682 14.768 1721
5 35 16.403 4531 11.754 3297 10.545 2437 12.974 2485
6 42 14.26 5126 10.775 3823 10.063 2986 12.762 3050
7 49 12.246 5690 9.684 4409 9.456 3792 12.116 3675
8 56 11.594 6040 9.547 4680 9.416 4105 12.789 3984
9 63 11.345 6350 9.472 4909 9.407 4382 13.195 4258
10 70 11.443 6600 9.443 5045 9.393 4495 13.521 4454
11 77 11.838 6722 9.423 4987 9.35 4176 13.822 4484
12 84 12.603 6922 9.438 4997 9.338 3881 13.912 4614
13 91 13.822 7216 9.455 5212 9.341 4069 13.845 4927
14 98 14.887 7467 9.462 5382 9.406 4197 13.79 5202
15 105 15.88 7657 9.524 5405 9.466 3860 13.782 5368
16 112 17.007 7949 9.555 5667 9.48 4118 13.742 5729
17 119 17.989 8220 9.729 5883 9.532 4225 13.719 6058
18 126 18.985 8524 9.912 6318 9.538 5171 13.689 6506
19 133 19.876 8817 10.213 6711 9.546 6121 13.679 6915
20 140 20.798 9206 10.451 7628 9.451 9695 13.656 7564
21 147 21.407 9310 11.146 7599 9.496 10181 13.658 7662
22 154 21.972 9312 11.815 7213 9.481 9403 13.652 7583
23 161 22.535 9253 12.385 6420 9.45 6230 13.641 7359
24 168 23.403 9518 12.633 6822 9.634 8122 13.722 7772
25 175 24.246 9852 12.907 7664 9.759 13353 13.816 8338
26 182 24.961 10082 13.317 8219 9.815 18960 13.888 8717
27 189 25.588 10179 13.809 8191 9.874 23016 13.945 8842
28 196 26.149 10106 14.327 7022 9.97 20098 13.987 8602
29 203 26.682 9907 14.817 4979 10.058 8077 14.021 8098
30 210 27.356 10004 15.163 4293 10.275 3217 14.147 8191
31 217 27.994 10029 15.526 3260 10.458 5 14.273 8141
32 224 28.606 10008 15.897 2057 10.593 0 14.403 7988
33 231 29.288 10288 16.197 1406 10.805 0 14.605 8414
34 238 30.049 11224 16.424 2507 11.136 5 14.89 10194
35 245 30.695 12135 16.753 4175 11.327 5 15.114 11949
36 252 31.235 12830 17.201 6423 11.431 0 15.28 13307
37 259 31.778 13971 17.604 15282 11.58 19 15.46 15566
38 266 32.291 15457 17.997 33371 11.721 116 15.629 18585
39 273 32.643 15485 18.644 43114 11.767 208 15.703 18756
40 280 32.981 14832 19.257 47503 11.856 59 15.771 17550
41 287 33.46 16165 19.59 85646 12.091 247006 15.957 20332
42 294 33.786 14412 20.15 92350 12.242 418886 16.029 16938

The following figure depicts model performance, in terms of RMSE, across various change-points for each of the methods evaluated:

Linear Models

The following figure depicts the optimal linear model along with the 4 other nearest change-points on either side of the optimal change-point

Quadratic Models

The following figure depicts the optimal quadratic model along with the 4 other nearest change-points on either side of the optimal change-point

Cubic Models

The following figure depicts the optimal cubic model along with the 4 other nearest change-points on either side of the optimal change-point

Exponential Models

The following figure depicts the optimal exponential model along with the 4 other nearest change-points on either side of the optimal change-point

Bootstrapping CP Approach

This section summarizes results using counts of SSD-related visits.

The following figure depicts the in-sample and out-of-sample performance (MSE) of various bounds on the opportunity window and different trends.

The following table depicts the top 10 specifications based on either aggregate or k-fold out-of-sample performance:

Aggregate Out-of-Sample
K-Fold Out-of-Sample
rank Weeks Days Model MSE Weeks Days Model MSE
1 7 49 Cubic 113.56 7 49 Cubic 191.86
2 8 56 Cubic 114.15 8 56 Cubic 192.30
3 9 63 Cubic 114.80 9 63 Cubic 193.06
4 9 63 Quadratic 115.09 9 63 Quadratic 193.29
5 11 77 Cubic 115.35 8 56 Quadratic 193.42
6 8 56 Quadratic 115.37 11 77 Cubic 193.59
7 10 70 Quadratic 115.57 10 70 Quadratic 193.68
8 10 70 Cubic 115.65 10 70 Cubic 193.85
9 12 84 Cubic 115.85 12 84 Cubic 194.13
10 13 91 Cubic 116.08 13 91 Cubic 194.36

The following figure depicts the observed and expected trend for the top 9 models based on 99-fold out-of-sample performance:

Summary of Model Performance Across Change-points

The following table summarizes the out-of-sample and 99-fold performance (RMSE) along with the implied number of missed opportunities for each method across the different change-points evaluated

Change Point
Linear
Quadratic
Cubic
weeks days Out-of-sample RMSE K-fold RMSE N Miss Visits Out-of-sample RMSE K-fold RMSE N Miss Visits Out-of-sample RMSE K-fold RMSE N Miss Visits
1 7 1037.57 1102.19 1096 415.36 488.33 678 204.95 280.41 372
2 14 734.94 801.45 2090 312.11 385.01 1351 178.22 253.15 800
3 21 539.62 614.48 2957 249.97 327.28 1973 163.9 241.22 1223
4 28 384.62 461.99 3796 196 274.38 2660 147.75 225.6 1814
5 35 268.54 349.56 4571 150.99 231.57 3365 128.85 208.38 2549
6 42 208.43 288.52 5173 131.15 211.25 3910 121.12 200.59 3150
7 49 165.86 243.34 5718 116.48 194.86 4448 113.56 191.86 3866
8 56 152.35 229.12 6068 115.37 193.42 4712 114.15 192.3 4173
9 63 143.23 220.44 6382 115.09 193.29 4945 114.8 193.06 4473
10 70 137.91 214.88 6644 115.57 193.68 5103 115.65 193.85 4661
11 77 138.87 216.09 6734 116.29 194.41 4898 115.35 193.59 4001
12 84 135.95 213.6 6957 116.81 195.03 4956 115.85 194.13 3851
13 91 131.02 208.87 7260 116.47 194.72 5202 116.08 194.36 4117
14 98 129.82 207.98 7493 117.92 196.18 5308 117.69 195.9 4069

Linear Models

The following figure depicts the optimal linear model (based on 99-fold performance) along with the 4 other nearest change-points on either side of the optimal change-point

Quadratic Models

The following figure depicts the optimal quadratic model (based on 99-fold performance) along with the 4 other nearest change-points on either side of the optimal change-point

Cubic Models

The following figure depicts the optimal cubic model (based on 99-fold performance) along with the 4 other nearest change-points on either side of the optimal change-point